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ABCTRACT 



A method and apparatus for displaying graf^cs data and 
video data, such as a video window, on a oon^>uter display. 
A graphics adapter diip stores graphics pixel data in a 
gr^)hics memory, and a video source stores video pixel data 
in a video memory. The graphics and video memories 
scqucntiaUy output blocks of pixel data to the display screen 
on output channels. Graphics data is selectively outpuaed on 
a number of graphics channels and video data is selectively 
ou^tted on the same number of video channels. The video 
channels arc multiplexed with the grsqihics channels to form 
the output channels and either graphics data or video data 
can be ou^t to the display on each output channel. A 
number of dummy video pixel values can be inserted before 
video data in the video memory to align video pixels 
between blocks of gr^hics data on the display screen. In 
one embodiment a source selection element selects graphics 
and video channels by reading a window-type memory to 
determine which pixels on the saccn are intended to display 
grap^cs data and which pixels are intended to display video 
data. The data on the output channels is converted to a form 
suitable fcs driving &e display screen of the computer 
system 

32 Claims, 8 Drawmg Sheets 
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METHOD AND APPARATUS FOR 
SIMULTANEOUSLY DISPLAYING GRAPHICS 
AND VIDEO DATA ON A COMPUTER 
DISPLAY 

BACKGROUND OF THE INVENTION 

The present invenUon relates generally to an apparatus 
and method for displaying graphics an a co^^)uta di^lay 
screen, and m<xc particularly to simultaneously displaying 
data from graphics and video sources on the same screen. 

Graphical 2^1ications are becoming increasingly popular 
with computer users. High resolution pictures, animation, 
and other visual and graphical effects displayed on a com- 
puter screen have become commonplace as computer micro- 
processors arc developed having greater speed and process- 
ing power. Graphical user interfaces (GUTs). for example, 
are used widely. It is gcncraliy accepted that CQnq>uters 
having graphical user interfaces (GUI^s) arc easier to use. 
and that it is (quicker to learn an application program in a 
GUI environment than in a non-GUI environment. 

The increased graphical c^abilities of computers have 
led to the display of video signals on computer display 
screens. A video source, such as a video camera, television 
receiver, etc. can be used to input a video signal to a 
computer. Conqwnents such as an analog-to-digital con- 
verter (ADQ convert the analog video signals to digital 
signals which the computer can process. These digital sig- 
nals are typically organized into •'pixels." which are funda- 
mental picture elements of an image on a display screeit The 
computer eventually sends the digital video data to a digital 
to analog converter (DAC) to display analog video signals 
on a computer display screen which look like the displayed 
images and animations of a CRT of a television, for 
example. Many computers can display live video signals In 
full color and resolution with no loss in frame rate or detail 

A common ^iplication of computer-displayed video sig- 
nals is to display a video ^Vindow"* on a screen suiroundcd 
by displayed computer-generated gr^>hics. For example, in 
a GUI. the comptiter typically displays a graphical 
background, several menu selections, icon shapes, open 
windows on the screen, etc A live video window can be 
di^layed on one portion of the screen while the rest of die 
screen di^lays standard graphical objects. A computer user 
could thus watch a live video window while working with 
other computer plications, such as a word processing 
window or a spreadsheet window. The size of the video 
window can be set by the user in some applications, 
although the resolution and frame rate for a certain window 
size dq)ends on the ^>eed and processing power of the 
computer and its display circuitry. 

In displaying both graphics and video on a computer 
screen simultaneously, the conq>utcr typically uses mcmoiy 
to store the graphics and video data before ou^tting the 
data to the screen. FIG. 1 is a block diagram of a typical prior 
art display system 10 used in a computer to simultaneously 
display video data and graphics data on the same screen. 
Ccmmands from a microprocessor arc sent on a system bus 
12 to a gr^hics adapter chip 14. which can be implemented 
as an application specific integrated circuit (ASIC). Graph- 
ics adapter chip 14 receives the conunands such as drawing 
commands, rendering conmiands, or commands to transfer 
data in niem<^ and performs those commands. Graphics 
adapter chip 14 sends generated data on bus 15 to VRAM or 
other types of memory chips 16 to store the generated 
gr^hics data. Gr^>hics data from die VRAM chip 16 is sent 
to digitai-to-analog converter (DAC) 18 when In^cted by 
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gr^>hic$ adapter chip 14. DAC 18 converts the digital 
gr^hics data to analog data to be displayed on a diq>lay 
screen 19. A common method of dis^day is to ou^t separate 
red, green, and blue (KGB) signals from the DAC to a color 

s display screen. A video window 21 is shown di^layed on 
di^lay saeen 19 with a graphics background 22. 

Display system 10 also includes a video source 20 for 
inputting a video signal. Video sources such as a video 
can^a, a video cassette recorder, or a television receiver arc 

10 typically used. The analog video signal from the video 
source is mpvx to an analog-to-digital converter (ADCV 
decoder/scaicr 24. which converts the analog video signal to 
a digital signal suitable for use with the other digital com- 
ponents of the system and extracts usable video data and 

1^ synchronization signals from the digitized video data. The 
ADC/decodeiyscaler 24 outputs digital video data, synchro- 
nization signals and other data derived from con^x>nents 
well known to those skilled in the art on bus 26, which is 
mixed with bus 15 output from graphics adq)ter chip 14. The 

^ video data is stored in VRAM 16 and thus shares the 
memoiy with graphics data generated by graphics adapict 
chq) 14. Typically, video data is stored in a particular section 
of memozy and is readily accessible by the graphics adaptex 
chip 14. (iraphics adapter chip 14 receives inf<»mation from 

^ the miac^ocessor indicating where the video window is 
located on the screen and causes grai^cs data oi video data 
to be output from VRAM 16 when a{^opriate. 

The prior art display system shown in FIG. 1 is useful in 
that a displayed video window can be stored in existing 

^ memoiy that is also used for graphics data, thereby obviating 
any need for additional memory and cost However, this 
display system is limited by the bandwidth of the memory 
bus 15. Since graphics data and video data ^are the same 
bus, the amount of graphics and video dau that can be stored 
in VRAM 16 and transferred to DAC 18 at one time is 
substantially reduced, especially when displaying **true 
color^ 24-bit video pixels, which require a large memory 
bandwidth. The perfonnance of the display system is thus 
reduced, and either the displayed video window is limited to 

^ a small size or a low resolution so that tbe full frame rate of 
the video signal can be displayed, or the frame rate of the 
video signal is reduced so that a particular resolution or 
window size can be displayed. In either case, the presenta- 
tion of a video signal in the video window is degraded 

What is needed is a display system of a computer system 
that provides a large mcm<»y buidwidth capable of display- 
ing a large video window at a high resolution and full frame 
rate while displaying gr^shics on the other portions of the 

^ display screen. The video window should be displayed 
without any video image clipping and without any restric- 
tions as to its displayed location on a cont^uter screen. 

SUMMARY OF THE INVENTnON 

55 To achieve the foregoing and other objects and in accor- 
dance with the purpose of the present invention, a method 
and apparatus is disclosed for simultaneously diqilaying 
gr^hics data and video data on a display screen of a 
computer. Separate graphics and video memories are used 

^ such that memory bandwidth and data transfer rates are 
higher, leading to larger displayed video window sizes and 
more realistic video presentation. The present invention uses 
dummy video pixel insertion in one embodiment to prevent 
losing video pixels and/or restricting the placement of tbe 

£5 video window on the computer screeiL 

A method is disclosed for simultaneously displaying 
graphics data and video data on a display screen of a 



05/12/2003, EAST Version: 1.03.0002 



5.896.140 

3 4 

conqnitcr system. The con^utcx system includes graphics memory. In an alternate cmbodimcDt of the present 

mcnsory and video memory arranged to store image infer- invention, a digital to analog controller can be used to select 

mation to be displayed on the display screen. The graphics gr^>hics and video lines to ou^t graphics and video pixels 

memory and the video memcry each sequentially transmit stored in a queue. 

blocks of data to the display screen on output dianncJs, and 5 The present invention penniis an increased amount of 

each block of daU includes data for multiple saeen pixels pi^e) data to be transferred from memory to Ihe dispUy 

that are transmitted simultaneously. The method includes ^^^^ ^ Separate graphics and video memories 

steps of storing graphics data received from a graj^cs ^^n each output a larger bandwidth of data. With an 

source in graphics memory and storing video data received inaeased data transfer rate, a larger and more realistic video 

from a video source in video memory. When only graphics 10 window can be displayed on the computer screen. 

data is to be Presented on toe screen in a Wock of screen ^ embodiment of the present invention also 

pixels, a block of graphics data is tra^tted o the saeen ^ ^ ^ ^ ^ ^ 

over a number of graphics channels. When only video data ^^^.^ ^ ^^^^^ 

15 to be presented on the screen, a block of video data is ^ ^^"^^ ^^^^ ^ video^pSSs. 

output trom the video memory on a nuinfier 01 video 15 ^ '^i^^^ 

to be placed on 

channel corresponding to the i.umb«aferaphi« channels. ^ ^ ^ ^ 

The video channels are coupled to the grqihics cfaaonels to ' j r i 

fonn output channels, and either graphics data or video data '"'"e and other advantages of the present inventioD will 

can be output to a diq>lay screen on eadi ou^nit channel parent to those skiUed in the art after rwding flie 

When bodigr^cs data and video data are to be transmitted 20 folo^'i^ descriptions and studying the vanous figures of 

to the screen simultaneously in a single block of data on the ^ drawings. 

output Aannels. the ou^ut channek that are intended cany DESCRIPnON OFTHE DRAWINGS 

graphics data are selecuvely caused to transmit only graph- ^r^^^ k^e^^k^m^ i^vi 

ics data. Likewise, die output channels that are intended to piG. 1 is a block diagram cf a prior art conqsutcr display 

carry video data are selectively caused to transmit only 25 system; 

video data. In one embodiment, the video memay stores a HG. 2 is a block diagram of a first cmbodimoit of a 

row of ou^ut data in a shift register befOTc the row is output conmutcx display system in accordance with the present 

A portion of the output data is shifted into an output buffer invention- 

before the data is output In one embodiment, to align the * . . ... •„ ^ ^. , ^ 

video window between a block of graphics data, a nimbcr 30 ^ ^ j^^f^ diagnun dlustratmg mulUplexcd 

of dummy video pixel values are inserted before video data f^^^ ^^P^^ 

in the shift register of the video memory. The number of • 

dummy pixel values is based on tfie position of the edge of ^G- 26 is a schematic diagram illustrating a connection 

the video window and the number of output diannels. The of » single graphics d»annel and video channel as shown in 

selection of graphics and video channels includes reading a 35 2b; 

window-type memory to detennine which pixels on the FIG. 3a is a diagrammatic illustration of a portion of a 

screen are intended to display grq)hics data and which display screen showing graphics and video pixels of a 

pixels on said screen are intended to display video data. displayed video window positioned at a discrete boundary 

An apparatus for displaying a video window on a display between gr^hics pixel blocks; 

screen of a computer system includes a griq>hics memory 40 FIG. 36 is a diagranunatic illustration d a portion of a 

having a set off output graphics channels suitable for simul- display screen showing a boundary between graphics and 

taneously transmitting graphics data for a plurality of screen video data that occurs at an intermediate location within a 

pixels. The apparatus also includes a video memory having pixel block; 

a set of ou^ut video diannels suitable for simultaneously HG. 4 is a block diagram of the dummy pixel logic of the 

transmitting video data for a plurality of screen pixels. Each 43 pf^ent invention* and 

video .^nel is coupled to a corresponding paphics dian- ^ 5 ^ ^ ^ ^^^^^ embodiment of 

f • *° *^ H " " computer display sy^m of the present invention, 

coupled to each of the pairs of graphics channels and video ^ r ^ i~ 

channels. A selection element is used to selectively caiise DETAILED DESCRIPTION OF THE 

data from the graphics channels (H^ the video channels to pass so INVENTION 

on each of the ou^t channels. The ou^t channels may 

transmit a block of pixel data that simultaneously includes FIG. 2 is a block diagram of a first embodiment of a 

both graphics data and video data divided on a discrete pixel computer display system 3* diat incorporates the present 

basis. A converter dement is used for converting data on the invention to display a video window on a display screen, 

output channels into a form suitable for driving the display 55 Display system 30 includes a microprocessor 31, a system 

screen of the c(Hnputer system. In one embodiment, the bus 32. a graphics adapter chip 34, graphics memory 36, 

selection element includes window-type memory having a window-type memory 38, a video source 40. analog-to- 

memory map of the location <rf a video window on the digital converters (ADC's) 42. a dccodcr/scalcr 44, a video 

computer screen. In the preferred embodiment the video memory 46, source selection logic 48. digital-to-analog 

memory stores a number of dummy pixels positioned before 60 convcrto' (DAC) 56. firame-grabbcr controller 52. and a 

the video data in the video memory. These dummy pixels are display screen 54. While buses of specified width (i.e.. 8-bit 

copied values of actual video pixels and are output on video buses. 32-bit buses, etc) arc dcscnbod below as an example, 

chaimels that are not seleaed to output data to the coraputo- it should be appreciated that a variety of different types of 

screen. Only dummy pixels are thus thrown away, and actual buses having different numbers of lines can be used in 

video data is not lo^ A fi-ame grabber controller is prefer- 65 different embodiments. 

ably coupled to the video menK)ry and the source element Microprocessor 31 is the main processor of the computer 

for controlling the ou^ of video data from the video system and is coupled to system bus 32 for data transfer to 
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components and peripherals of the system, including di^lay 
system 30. System bus 32 is also coupled to RAM. ROM. 
input/ou^t poits. and other oompooents geoeraily used in 
a computer system (not shown). System bus 32 can be used 
to send data signals, address signals, and control signals, s 

Graphics ad^er chip 34 receives data over system bus 32 
and creates graphics pixel data to be displayed on a display 
screen. Commands from the microprocessor of the computer 
system are input to the gr^hics adapter chip on the system 
bus and instruct the chip to draw graphical objects, math- 
ematically render images, etc, and to aeate pixel data to be 
displayed on the display screen. A suitable graphics adapter 
chip for use in the described embodiment is a Sun GX a 
TGX aj^lication ^)ecific integrated circuit (ASIC), manu- 
factured by Sun Microsystems. Inc. of Mountain View. ,5 
CaSlf. This ASIC includes a graphics rendering engine, a 
memory controller, and a C^KT/di^lay controller on a single 
chip. Sq)arate chips having these functions can be used as 
weU. Graphics adapter chip 34 outputs graj^caJ data on bus 
55. which in the desiaibed embodiment is a 64 bit bus. Bus 
55 is divided into two 32-bit wide buses which ii^nit data to 
gr^)hics memory 36. The graphics data is preferably for- 
matted into a number of pixels, wherein a pixel is the 
smallest displayed picture element on the display screen. 
Taken collectively, the pixels form an image. Pixels are 
generally positioned in rows and columns on the screen. 
Each pixel is represented by a number of bits, and the 
numerical value of the bits indicates a pixel's attributes, such 
as the color or shade of the pixel. 

The graphics adapter chip controls the display of graphics 30 
pixels by continuously outputting stored pixels from graph- 
ics mcnK)iy to DAC 50. Graphics adapter chip 34 also 
receives inTormation from die microprocessor indicating 
where a video window is to be displayed on the display 
screen. This information, is input to window-type memoiy 35 
38 (described below). 

Gr^hics memory 36 receives graphics data from graphics 
ad^er chip 34 and stores the data until it is output and 
displayed on the screen. In the described embodiment, two 
banks of memory 58 and 60 are used to store the graphics 40 
data, where the Arst four of any eight horizontally contigu- 
ous pixels are stored in one bank 58. and the second four 
pixels are stored in the second bank 60. Graphics pixels 
stored in graphics memory banks 58 and 60 have eight bits 
^iece in the described embodiment which arc used to store 45 
information about the color of the displayed pixel. In the 
described embodiment, each bank 58 and 60 is a 128Kx8 
VRAM chip. In alternate embodiments, other types of 
memory can be used as well, including dynamic random 
access memory (DRAM). Also, in alternate embodiments. 30 
the graphics pixds can have greater or fewer than 8 bits per 
pixel, such as 16-bit or 24-bit pixels. If a pixel includes more 
than 8 hits, a larger graphics memory than the described 
memory would typically be required to store the pixels, and 
a greater width the datapath between graphics memory ss 
and DAC 50 would be needed. In addition, a DAC 50 
supporting the larger pixds and multiplexing would be 
needed. 

Graphics memcwy 36 outputs data on 32-bit bus 62 in the 
described embodiment. The outputting of graphics pixels is 60 
controlled by gr^hics adapta chip 34 using address and 
control lines (not shown). The graphics adapter chip con- 
tinuously outputs graphics pixels at a rate consistent with the 
diq^ay screen's refresh rate. In die described embodiment, 
the data on bus 62 is divided into 4 pixels of 8 bits per pixel. 65 
for a total of 32 bits. Each 8 bits dT bus 62 is considered a 
**graphics diannd*" that carries information about 1 pbcel at 
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a time; bus 62 therefore indudcs four graphics channels in 
the described embodiment. The four graphics channels 
transmit four pixds sinuiltancously from the graphics 
mcmoiy; hcrdn. these four pixds are oonsido-ed a '1>lock" 
of graphics data. **>r graphics channels can be used in 
alternate embodiments, where *^ can be a value of 2 ot 
greater. 

Each bank 58 and 60 of graphics memoiy 36 includes an 
output buffer which ou^>uts a single 32-bit block of four 
pixels at a time. When using a VRAM or similar type of 
memory, shift registers in ffapbics memory 36 can be used 
to provide pixds from the memory storage locations to the 
output buffer, as described below with reference to video 
memory 46. Banks 58 and 60 also indude tri-state buffers 
for each output bit. These tri-state buffen can be enabled or 
disabled to selectively output bits at specific locations in the 
output buffer. Source sdection logic 48 can control the 
tri-state buffers to select which graphics channels are 
enabled to output graphics pixels (described below). The 
graphics channds are multiplexed with video channels from 
a video memory and seat to a DAC to be output to the 
display saeen, as described t^elow. 

Window-type memoiy 38 is coujded to graphics adapter 
chip 34 via bus 55. In the described embodiment, window- 
typc memory 38 recdves the lower 4 bits of bytes output by 
gr^hics adapics dhip 34 on a bus 63; since grf^cs adapter 
ch^ 34 preferably outputs 8 bytes, window-type noemory 38 
recdves 32 hits. The microprocessor 31 writes data into 
window-type noemory 38 through the graphics ad^ter chip 
34 indicating the pixel layout of the display screen, i.e.. 
which pixels on the display screen are graphics pixels and 
which pixds are video pixds. The window-type memory 
stores the screen layout as pixel codes indicating a pixel- 
by-pixel description of the display screen. For exan^le, the 
microprocessor can store a descriptioo of the location of the 
video window on the saeen in the window-type menkory. In 
this description, video pixds can be indicated by a oeitain 
pixd code and graphics pixds can be indicated by a different 
pixd code. In (he described embodiment window-type 
memory 38 is four bits deep; thus, a video pixd can be 
indicated by a pixd code of 15. and a graphics pixd can t>e 
indicated by a pixd code between 0 and 14. The codes 
between 0 and 14 can contain other data related to the 
display screen layout. For exan4)le. data related to color of 
pixds for each specific application program that is running 
on the conq)uter system can be stored, such as colc»' palette 
information (described below). In alternate embodiments, a 
snoaller window-type memcny can be used, since pixd codes 
of only 1 bit per pixd are actually required to store pixds as 
either video or graphics pixels in the window-type memory. 

The data in window-type mem(Ry 38 is accessed by the 
source selection logic 48 through bus 61. which is preferably 
a 16-bit bus. Four 4-bit pixel codes (one pixel f<n each 
chanod) is supplied to the source sdection logic on bus 61. 
Bus 61 is also coupled to DAC 50 to supply the four pixel 
codes to die DAC from window-type meooory. For example, 
when die DAC 50 reads a value of 15 on bus 61 from the 
window-type memory data, it expects a 24-bit video pixd 
(from video memory), and when the DAC reads a value 
between 0-14 on bus 61. it expects an 8-bit pixel (from 
graphics memoy). In the described embodiment, window- 
type memory is the same size as each bank of graphics 
memoiy 36. 128Kx8. 

Video source 40 is used to input a video signal into the 
di^lay system 10. \^deo source 40 can be a television 
rccdvCT, video camera, video cassette recorder, or any other 
apparatus capable of producing an analog video signal 
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Video signals typically arc produced in one of two types. be clocked into several ports simultaneously if desired; this 

One type, composite video, includes a channd on which a is accon^lisbed to inscn *'dummy pixels'* in the video 

brightness signal, a color signal and various syndironization memoiy and is described in greater detail with reference to 

signals are canied. Con:4X)site video signals have various FIGS. 3o. 36. and 4. The video memory of the described 

fonnats. including NTSC. PAL. and SECAM. The other type 5 embodiment is 1024x5 12x24. so that the maiimum-sized 

of video signal is S-video. v^ch includes a luminance video window having 24-bit pixels stored in the video 

signal and a chrominance signal. In the described memoiy is 1024x512 pixcb. In alternate cmbodimcDts. 

embodiment, either type ctf video signal may be input to the diffcrcntly-sizcd video windows or video memories can be 

display system 10. ADC's 42 are used to convert the analog used; for cxampie, two identical 5 12x5 12x24 storage areas 

input video signal to a digital signal which can be manipu- |q of the video memory can be used to reduce the ^ideo 

latcd by the display computer system. In the described tearing" effect, which occurs when computer monitor 

embodiment, a TDA8708 for composite video (and part of refresh rates differ widely from video source refresh rates. 

S-video) and aTDAS709 for S-video. both manufactured by Ihc two identical storage areas in memory can be used as 

Philips/Signetics, arc used as ADC*s 42. In alternate two buffers of video memory in which video pixels are 

emtx>diments. a video source can provide a digital video 15 stored. The two buffers can alternately ou^iut a complete 

signal directly so that no ADC 42 is required. frame <^ video data as the buffers arc filled so that ik> partial 

The ADC's 42 ou^t a digital signal on buses 64. In the frames; arc displayed on the screen, 

described embodiment busses 64 are 8-bit buses that send Video memory 46 of the described embodimem includes 

a digital signal representing the analog video input signal a second sequential (or serial) access memory port used for 

Two buses 64 are provided in the described embodiment 20 outputting video pixels from the video memory. The output 

eadi bus corresponding to a type of video signal (e.g., channels are organized into three buses of 32 hits each, 

composite or S-video). Buses 64 are coupled to decoder/ where eadi group of 32 bits is used for one primary color of 

scaler 44. which decodes the 8-bit input signal to a 24-bit red. green, or blue. Since 8 bits rei^sents each primary 

red-green-blue (RGB) output signal. A commonly-used color in 24-hit video signals, each of the 32-bit buses 

pixel format for digitized video is a 24-bit RGB format 25 iiidu<*es four video channels of eight bits each. Each 32-bit 

where each of the three primary color portions (red, green bus thus can send one color component (R. G. or B) of four 

and blue) is described by 8 bits. 24-bit pixel format is video pixels, one video pixel on each video charmcl. A. 

considered 'Irue color", since a much larger and realistic described below, DAC 50 receives 24-bit video pixels 

range of colors can be displayed than if only 8 bits were separated into the three RGB 8-bit portions. Bus 74 thus 

used. Other N-bit pixel fonnats can also be used in alternate 30 carries the 8-bit red components of 4 video pixels, bus ?• 

embodiments; for exan^le. 16-btt or 15-bit RGB (nxels and carries the 8-bit green conqx>nents of 4 video pixels, and bus 

16-bit YUV pixels are common formats. The decoder 72 carries the 8-hit blue components of 4 video pixels. As 

extracts the synchronization signals from the digital video referenced herein, a "block" of video pixels refers to the 4 

signal and converts the raw digital data into standard luml- video pixels output on buses 74. 70. and 72, 

nance and duominance signals in the RGB format The 3s In the described embodiment, buses 70 and 72 are coupled 

scaler of decoder/sealer 44 also scales the input digital data to Green and Blue inputs of DAC 50. respectively, while bus 

from a standard, predetermined resolution that is automati- 74 is multiplexed with graphics channels 62 from graphics 

cally decoded by die decoder to a resolution specified by the memory 36 and coupled to die Red input of DAC 50 

miaoprocessor. For example, if a user resizes the video (described below). Since bus 62 and bus. 74 each include 

window on the display screen, the microprocessor sends the 40 four channels, the graphics channels and video cfaaimels 

new video window size to the fr^e grabber controller 52 on employ 4-way multiplexing. In alternate embodiments, 

system bus 32. The frame grabber controller then sends the N-way multipieung can be employed, where N can be a 

video wiiKiow size information to scaler 44 on bus 65. Scaler value of 2 or greater. 

44 adjusts the resolution of the video window and outputs When video pixels are output from the video memOTy 46, 

video pixels that corre^nd to the new video window size, 45 a row of video pixels is loaded into shift registers (or serial 

as is well known to those skilled in the art In the described access memory) included in the video memory from banks 

embodiment, the scaler scales ^e video window from a A-D. For example, a row can equal 1024 pixels in a 

standard resolution of 640x480 pixels to the desired reso- horizontal scan line. A block of video pixels is then shifted 

lution. Decoder/sealer can be iiiq>lemented using model from the shift registers into an output buffer that stores a 

SAA7196 manufactured by Phlllips/Signetics. In the 50 block of video pixels and which is also included in video 

described emt>odiment. 24-bit RGB video data is output memory 46. In the described embodintent. a block of video 

from deooder^scaler 44 on bus 66. data includes 4 video pixels, one pixel per video channel 

Video mem(My 46 is coupled to bus 66 output from where each Bank is coupled to one video channel. The 

decoder/sealer 44. In the described embodiment bus 66 is output buffer, similar to the output buffer described in 

routed into four buses of 24 bits eadi. where each bus is 55 gra{^cs memory 36. outputs a single block of pixel data at 

input into a different bank ofvideontemory 46 (Banks A-D). a time for three 32-bit buses, thus totalling 96 bits. In 

Banks A-D are incorporated in a sequential access memory addition, video channels are coiq>led to the ouq>ut buffer of 

port of ttie video memory (all four banks are shown as one video memory 46 and are controlled by tri-state buffers 

box 46 in FIG. 2). The data at each bank of the video which either enable a video channel to carry data or disable 

memory is clodced in sequentially and is stored in the same 60 the video channel and prevent data from being output on bus 

order In the memory (the dau is preferably docked into siuft 74. The tci-state buffers <^ bus 74 are controlled by source 

registers within the video memory and then stored from the selection logic 4S. which is described t>elow. Preferably, one 

shift registers to a DRAM portion (banks A-D) of the video block of video pbtels is output from the output buffer, and a 

memory. The shift registers are described l>elow). Thus, die new block is shifted from the shift registers to the oittpm 

decoder/sealer 44 sends out a 24-bit pixd which is docked 65 buffer. Once the shift register has been exhausted, the next 

into Bank A. another video pixel whidi is docked into Bank row of pixels is loaded into the shift register from Banks 

B. and so on. A video pixel output by decoder/sealer 44 can A-D. 
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Video memoiy 46 of the described embodiment also 
preferably includes a random access port coupled to a bus 76 
which is connected to the microprocessor through the sys- 
tem bus 32. The random-access port can t>e used to randomly 
access the contents of video memory 46. An aj^cation dial 
uses such a configuration is teleconferencing, in which a 
video signal depicting, for example, a user's face is sent to 
the miaoprocessor and transmitted through a networking 
interface to another computer/display screen. A user could 
thus receive a video signal from a different user to display 
on the computer screen while the user*s own picture would 
be recorded by a video camera near the screen and sent to the 
other user*s conq)uter for that user to view on his screen. 
When outputting video data from the random access port of 
video memory 46. a buffer 47 is used to reduce loading on 
microprocessor bus 32. 

Video memory 46 is a iri-port video memoiy. such as 
MT43C8128 manufactured by Micron Scmiconductcx. The 
three ports arc the input sequential access port the ou^Hit 
sequential access port, and the random access port. In 
alternate embodiments, other types of video memory can be 
used. For example, a dual port video mcnK)ry can be used 
which includes one sequential access port and one random 
access port The sequential access port can be used for 
outputting the video pixels to the DAC 50. and the random 
access port can be used both as an input port for video pixels 
from dccodcrVscalcr 44 and as an ou^ut port to send video 
data to the microprocessor and over a network for tdecon- 
fcrencing. The dual port video memory is typically cheaper 
than the tri-port video memory, but it is slower due to the 
shared use of the random access port as an input and an 
output 

Source selection logic 48 is used to select when graphics 
pixel data and video pixel data are to be output to the DAC 
50 and displayed on die display soreen. The source selection 
logic moniton pixels codes that are caused to be output of 
window-type memoiy 3$ on bus 61 by graphics adapter chip 
34. The source selection logic receives the pixel codes via 
16-bit bus 61 in groups of four, where each pixel code 
identifies a grai^cs pixel or a video pixel. The pixel codes 
instnjct the source selection logic to turn ou^ts of the 
gr^hics and video memories on or off. The source selection 
logic sends enable or disable signals on bus 45 to Ihe tri-statc 
buffers at the output buffer of bank 58 of the graphics 
mentory 36, and sends similar signals on bus 49 to the 
tri-state buffers of bank 60 of graphics iDcmory 36. Some or 
all of die tri-stete buffers arc enabled when die window-type 
memory indicates which graphics channels should output 
gr^hics pixels, and die tri-state buffers which are not 
selected to output pixels are left in a high-impedance state. 
Similarly, the pixel codes instntct the source selection logic 
to send enable or disable signals oo bus 51 to the tri-state 
buffers of bus 74 at the output buffer of video memoiy 46. 
Td-statc buffers are enabled for video channels which are 
indicated to output video pixels. 

In the described embodiment, source selection logic 48 
provides the following signals on bus 51 to the video 
memory 46. determined by the following logic equations: 

EA=;/(WTA0&WTA1&WTA2&WTA3) 
EB=^(WTBaAWTBl&WTB2AWTB3> 
EC=/(WTOO&WrC l&WTC 2&WTC3) 
EIW(WTDO&WTD l&WTD2ikWTD3) 

where EA. EB. EC. and ED arc the (acdve-low) signals that 
enable or disable the tri-state buffers of the video memory. 
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• and. for exaiiq>le. WrA[0:3] are die four bits ftom window 
type memory forming a pixel code {**r indicates the inver- 
sion of the term in parentheses). In the described 
embodiment, if the four bits are all high, which is a pixel 

5 code of 15. then a vixleo pixel is indicated. The FIA signal 
is thus sent as an enable (low) signal to enable the tri-statc 
buffer for the corresponding output video channel of bus 74 
of video memoiy 46. If any of the Wts of the pixel code are 
low, then a disable (high) signal is sent to the corresponding 

10 tri-state buffer to disable that video channel of die video 
memoiy. WTB. WTC and WTD are the three other 4-bit 
pixel codes read on bus 61 to enable or disable the appro 
(»iate tri-state buffers of video memoiy 46. 

Source selection logic 48 provides the following signals 

IS on bus 45 to bank 58 of graphics memory 36: 

B0_JEA =BO_GREOKWTA0 & WTAl & WTA2 & 
WTA3) 

BO_EB =BO„GRE<^WTB0 & WTBl & WTB2 & 
WTB3) 

^ BO JC =BO_GRE^WrCO & WTCl & WTC2 & 
WTC3) 

BO_JED =B0_GREO+<WTD0 & WTDl & WTD2 & 
WTD3) 

^ The following signals are provided on bus 49 to bank 60 
of graphics memoiy 36: 

B1_EA?:B0_GRE1+<WTA0 & WTAl & WTA2 & 
WTA3) 

B1_EB=B0_GRE1-KWTB0 & WTBl & WTB2 & 
30 WTB3) 

Bl_EC=BO_GREl-KWrCO & WTCl & WTC2 & 
WTC3) 

Bl_EI>=B0_GREl-KWTDO & WTDl & WTD2 & 
WTD3) where BO_£A-D and B1_EA-D arc the 

33 (active-low) signals diat enable or disable the tri-statc 
buffers on bank 58, BO_GREO and B0_GLRE1 are 
enable signals from graphics chip 34 to enable cither 
bank 58 or bank 60. and WTA-D|0:3] are die four biu 
forming a pixel code. If the four bits for a channel are 

40 - not all high, and the BO_GREx signal is hi^. then a 
graphics pixel (pixel code of 0-14) is indicatnd. and the 
en^le signal, such as BO_EA, is set low to enable die 
coiresponding tri-state buffer and graphics dianncl of 
the indicated bank of graphics memory 36. If all of die 

AS bits are high . then the enable signal is set high to disable 
the tri-state buffer for that channel. In other 
embodiments, other logic can be used to achieve the 
same results. 

Source selection logic 48 also sends a frame grabber 

50 enable signal on bus 92 to frame grabber controller 52 on 
bus 92 to signal the frame grabber controUer to begin 
sequcndng video pixels out of video memory 46. The frame 
grabber enable signal is output from the source selection 
logic when the source selection logic receives a pixel code 

55 of 15 (all 4 high bits), indicating a Wdeo pixel, on any of the 
four channels output on bus 61 from window-type memory 
38. Source selection logic 48 also receives two signals from 
the frame grabber controller on bus 92 which indicate if a 
display update cycle has occurred and if, so. allow the frame 

60 grabber controller to control the video memoiy^s tri-state 
buffers for that cycle (described below). 

In an alternate embodiment, multiplexing logic can be 
used to select specific graphics and video channels instead of 
enabling or disabling memoir outputs, as Is well known to 

65 those s^ed in the ait 

Digital-to^analog converter (DAC) 50 is coupled to sev- 
eral input channels for the puipose of converting input 
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digital data to output analog signals used by tiie di^lay conncctioii between each grq)iiics bit line 84 in dianncl 76 

screen to display pixels. In the described embodiment. DAC with each video hit line 86 in channel 78. An ouQjut bit line 

50 is a Brooktrce BT463 DAC which includes an R input 88 is coupled to each graphics and video bit line connection 

a G input a B input, and a WT input DAC 50 preferably has Since cither graf^cs data or video data is displayed on the 

two modes to display gnqjhics pixels: a true color. 24-bit 5 screen, cither a graphics channel 76 or a connected 

display HMxlc, and a '^scudo-color" mode. The pseudocolor (concsponding) video channd 78 is "selected** to output 

mode is an 8-bit di^lay mode whidi uses a programmable data (i.e.. allowed to send its data) on the connected output 

color ^^pal^e"* stored in the DAC to display **pseudo** 24-bit dianncl 80. The method of sending graphics and video 

colors for graphics pixels. The color palette is a hardware pixels and selecting appr o pri ate chaimels is described below 

(softwaie-progFammablc) look-up table that matches 24-bit lO Referencing FIG. 2, frame grabber controller 52 is 

colors with the 8-bit graphics values that are input to the coupled to the microprocessor of the computer system by 

DAC in pscudo color mode. For example, an 8-blt graj^cs system bus 32. Frame grabba controller can be imple- 

pixcl value between 0-255 that is iiipist to DAC 50 is mcnted with an ASIC controller or with a diflfocnt archi- 

rcfcrcnccd on the color palette, which stores a corresponding tecturc: for example, a KiHnx 4000-series field program- 

24-bil value for each 8-bit value. The corresponding 24-bit i5 mablc gate array (FPGA) can be us,cd. The frame grabber 

value is used as the color of the 8-hit pixel The 24-hit values controller controls die sequencing of input video data and 

in the color palette are previously chosen to correspond to output video data to and from video memory 46. Bus 90. 

specific g-bit values. Typically, different color palettes are including address and control lines, is used for general 

used for different application programs executed by miao- control of the video menoory, including refresh, clocking 

processor 31 (or another connected microprocessor); for 20 video input video ou^t and miax^ocessor access, as is 

example, the active application program provides an indi- well known to those sidlled in the art Frame grabber 

vidual color palette from which aU displayed 8-bit pixels arc controller includes dummy pixel logic 53 for the insertion of 

referenced. When the active application program is changed. dumnmy pixels in the video memory, as described below, 

a new cqIgt palette with different colors can be loaded and Franx grabber controller 52 also contrds buffer 47 via 

used by DAC 50. Pseudo color mode has the advantage that 25 line 91. Line 91 controls when video data from the random 

the graphics pixels require less memory space and process- access port of video memory is sent on the system bus to the 

ing tinK.. with the drawback of less realistic pixel colors. microprocessor to be sent out on networks wires, such as in 

The mode of DAC 50 is selected for each pixel by the teleconferencing applications, 

pixel codes from window-type memory 38. Graphics Frame grabber controller 52 also receives signals from 

adapter chip 34 causes window-type memory 38 to send 30 and sends signals to source selection logic 48 on bus 92. As 

pixel codes to DAC 50. If the pixel code from the window- described above, bus 92 preferably includes 3 lines. One line 

type memory is a value between 0-14. indicating a graphics is used to carry a signal from the source selection logic 48 

pixel, the bits of the pixel code can be used to select either to frame grabber controller 52 indicating that video pixels 

true color (24-bit) mode or pseudo color (8 bit) mode. F<s should be output on bus 74. Irame grabber controller 52 

example, if the first bit of the pixel code is 0. one mode is 3S would begin to shift video pixel data out of video memory 

indicated, and if the first bit is 1, the other mode is indicated. 46 upon receiving this signaL 

This graphics pixel code information can also be used to The remaining two lines of bus 92 are used to send a 

selea different palettes for different gr^ihics pixels in display update signal from frame grabber controller S3 to 

pseudo color mode, as is well-known to those skilled in the source selection logic 48. The di^lay update signal allows 

art 40 the frame grabber controller to control the serial outputs of 

DAC 50 uses all three inputs R. G, and B for video pixels. video memory during a display update cycle or **blanking.** 

A video pixel is separated into three 8-bit portions, where A display update cycle occurs when a display device, such 

each portion is input to one of the RGB inputs. A video pixel as a CRT or display saeen, must stop displaying data so that 

is combined into 24 bits in DAC 50. The R input preferably the scan line can be reset from the right e<^e of the saeen 

receives the fir^ 8 bits, the G iiQ>ut the middle 8 bits, and the 45 to the left edge of the screen (*1retrace**) at the beginning of 

B input the final 8 bits of a video pixel. Each RGB input the next scan line. The frame grabber controller knows when 

receives 4 video pixels at once in the described embodiment the display update cycle occurs and so controls the video 

each video pixel being sent on an 8-bit video diaimel. The memory tri-state buffm to load data into the shift registers 

32-bit buses shown in FIG. 2 each represent four 8-bit video in the video memc^ during the display update cycle, as is 

channels. 50 well known to those skilled in the art The second line of bus 

The R input of DAC 50 is a special case. Since the 92 selects which logic is controlling the video tri-sute 

ffaptics pixels from gr^>hics memory 36 are 8-bits (in the buffers (either the source selection logic (e.g.. low) or the 

described embodiment), only the R input to the DAC 50 is frame grabber controller (e.g.. high)). Whra the output 

used f<x displaying graphics pixels in the described embodi- device is on a display update cycle, the frame grabber 

ment Since both graii^cs data and video data use the R 55 controller is selected. The third line of bus 92 is used to 

input, the graphics chaimels of bus 62 and the video chan- transmit a serial enable signal from die frame grabber 

ncls of bus 74 coupled to the R input are multiplexed. Either controller; this signal is used when the frame grabber 

a graphics pixel or a video pixel is sent to the R input of the controller has been selected by the second line of tnis 92. 

DAC on each diannet In other embodiments, different The serial enable signal allows the frame grabber c<»itroUer 

additional inptxts to DAC 50 can be similarly multiplexed. 60 to control the tri-state buffers during the display update 

As shown in FIG. 2a. each 8-bit gr^hics channel 76 of cycle, 

bus 62 is connected to a corresponding 8-bit video channel In ttie described embodiment, frame grabber controUer 52 

78 bus 74. Output diannels 80 of bus 82 are connected includes dummy pixel logic 53. Dummy pixd logic 53 is 

from the junction of the graphics and video channels to the used to control the clocking of eadi video pixel to video 

DAC 50. A block of output pixels includes the pixels on the 65 memory 46 from deooder/scaler 44. Logic 53 uses die 

four output channels SO. As shown in FIG. 2b, the connec- dummy pixel value to detcimine how many duimny pixels 

tion of each 84>it channel 76 and 78 indudes a similar are to be inserted in the input video pixel stream bdore the 
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first video pixel on a displayed horizontal scan Uae. The 
iDsertion of dummy pixels are described in greater detail 
with respect to FIGS. 3a and 3fr. and a prefecred implemen- 
tation of dummy pixel logic 53 is shown in FIG. 4. Dummy 
pixel logic is shown implemented on the same integrated s 
circuit chip as frame grabber controller 52. In alternate 
embodiments, dummy pixel logic 53 can be implemented as 
a separate chip or. for example, on the same chip as source 
selection logic 48. 

Display screen 54 is a standard computer monitor or lo 
similar display preferably capable of displaying high reso- 
lution graphical pictures. Display screen 54 is coupled to 
DAC 50 and receives analog RGB outputs which determine 
the colors of pixels displayed by screen 54. A video window 
57 of video pixels is preferably displayed amid a back- 15 
ground of gr£Q>hics pixels 59. 

The display s>'stem of FIG. 2 operates to display graphics 
data and video data on a display screen as follows. Micro- 
I^ocessor 31 sends instructions on system bus 32 to graphics 
adapter chip 34 to store a pixel code map in window-type 20 
memory 38 which indicates the current location of the pixels 
of a video window on the display screen. The microproces- 
sor sends instructions to the graphics adapter chip to gen- 
erate graphics pixel data in accordance with the miaopro- 
cessor commands. The graphics adapter chip stores the 25 
generated graphics pixels in both banks of grq)hics memory 
36. When the screen is ready to display pixels, the graphics 
adapter chip sends signals to gr^hics memory 36 to send 
out the graphics pixels in accordance with the display 
screen's parameters, such as refresh rate. The gra{^cs 30 
adapter chip continuously sends out graphics data from the 
graphics memory. 

FIG. 3a is a portion of a display screen showing video 
pixels in a video window 96 and graphics pixels 97 sur- 
rounding video window 96. The pixels on die display screen 35 
are typically displayed in horizontal scan lines from left to 
right. For a particular horizontal scan line (or row) of pixels 
98. the first pixel 100 on the left edge of the screen is 
displayed, followed by the next pixel 102 to the right, and so 
on. until the right edge of the screen is reached. The process 40 
then repeats with the next horizontal row of pixels 104 
below the horizontal row just displayed, etc. 

Graphics memory 36 normally outputs blocks of four 
graphics pixels at a time to DAC 50 in the described 
embodiment. During this process, the source selection logic 43 
48 enables all the iri-state buffers at the outputs of graphics 
memory 36 and disables all the tri-sutc buffers at the bus 74 
outputs of video mezncry 46 so that only graphics channels 
are selected to ou^ut graphics data (the video memory is 
storing incoming video pixels from decodcr/scalcr 44 at this 50 
time). The graphics pixels arc displayed sequentiaUy on the 
di^lay screen. Blocks 1€6 of four graphics pixels are 
displayed from left to right until the desired left border 108 
of the video window is reached. At this point the source 
selection logic 48 reads video pixel codes for the video 55 
window from the screen pixel map in window-type memory 
38 (i.e.. pixel codes having a value of 15 in the described 
embodiment). The source selection logic disables the tri- 
state buffers of all the graphics memory 36 outputs. Source 
selection logic 48 then signals the frame grabber controller 60 
52 on bus 92 with the frame grabber enable signal to begin 
sequencing blocks of four video pixds at a time out of video 
memory 46. At the same time, source selection logic 48 
enables tri-state buffers on the outputs of video mcnK>ry 46 
to allow video pixels on video channels 78 to be output from 65 
the video memovy to the DAC. Using this process, blocks of 
graphics pixels arc sent on output channels 80 (see FIG. 2a) 
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when die channels 80 are selected to diqiiay gr^rfiics data, 
and blocks of video pixels are sent on output chacmcls 80 
when die channels 80 are selected to display video data. 

The left border 108 of the video window 96 as shown in 
FIG. 3a is aligned widi die edge of a block of grq>hics data, 
i.e. a discrete graphics bdock boundary. The left border ctf the 
video window is displayed after a block 106 of four graphics 
pixels have been di^layed. In this situation, all four graph- 
ics channels are selected (enabled) to ou^t a block of 
grq>hics pixels on output channels 80 until the video win- 
dow is reached, at which time all the gr^ihics channel 
tri-state buffers are disabled and all four tri-state buffers on 
video memory 46 are enabled to output a block of four video 
pixels on all four video channels 78. At the right edge of the 
video window (not shown), the source selection logic reads 
a graphics pixel code from window-type memory 38 and. in 
response, disables the video charmels. signals the frame 
grabber controller to stop shifting out video pixels from 
video memory 46. and enables all four grsqdiics chaimels. 
The same process occurs for each horizontal scan line. 

FIG. 36 shows a different situadon. in which the left 
border 108 of the video window is di^layed between a 
grjq>hics block boundary. In this situation, somt the four 
ou^t channels 80 must be selected to output graphics 
pixels, and, simultaneously, some of the four output chan- 
nels 80 must be selected to output video pixels. That is. a 
block of data on the four output chaimels 80 must include 
t>oth graphics pixels and video pixels. In the example of FIG. 
V>. the fint two output channels must be graphics chaimels 
so dial graphics pixels HO and 112 are displayed, and the 
final two output channels must be video chaimels so that 
video pixels 114 and 116 are di^layed. 

To permit the video window to be displayed at an arbitrary 
graphics pixel boundary, video pixds must be ou^ut on 
sp^nfic video channels. The outputs of a video memory 
typically function by emptying out a Uock of all four video 
pixels (32 bits) in the output buffer of the video memory at 
once (for each 32 hit bus 70. 72 and 74). A video pixel is 
physically output on a video channel 78 only if the tri-state 
buffer for that video channel has been enabled. The next 
block of four video pixels in memory are then loaded into 
the output buffer of the video memory to be output at the 
next opportunity. Tliis creates a problem wheiL as shown in 
FIG. 3^. only some of the video pixels need to be ou^ut If 
only die third and fourth tri-state buffers arc enabled, the first 
and second video pixels in the ou^t buffer are lost and 
never seen in the video window. This loss of data creates a 
cropped or *'clipped** video window picture. On the other 
hand, if all four video pixels are output every time video 
pixels are displayed (no outputs disabled), then the video 
window cannot be arbitrarily aligned on any graphics pixel 
boundary; the video window must be displayed with hori- 
zontal alignment restrictions suc^ that in an N-way rmdti- 
plcxed systenL die left border of the video window can only 
be positioned at every Nth horizontal pixeL 

A solution to this arbitrary video window alignment 
problem in accordance with the present invention is to insert 
a number of dummy video pixels ("dummy pixels**) in the 
sequence of outputted video pixels to shift the actual video 
pixel data. In the exanQ}le of FIG. 36. two dummy pixels are 
inserted before tie block of video pixel data. This shifts tiie 
first and second video pixels of the block into the diird and 
fourth pixel locations (positions) of the block, the former 
third and fourth video pixels are shifted into the first and 
second portions of the next block of video pixels, and so on. 
When the block of video pixels is ou^utted from &xt output 
buffer of die video memory, die first and second video 
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cfaannds arc disabled and the data at the first and second 
pixel positions art lost However, only dummy data is in 
those first and second pixel positions. The third and fourth 
video channels are enabled, allowing the former first and 
second video pixels to be output and displayed on the screen 
in their proper sequence. 

The number of dummy video pixels required to place the 
video window on an arbitrary graphics pixel boundary 
without loss of video data is equal to the column nunober 
the left border of the video window modulus the number of 
ouqnit channels 80. In the example of FIG. 3^, the video 
window begins at cohimn number 202 (counting from the 
left edge of the saeen). There are 4 output channels in a 
4-way multiplexed system, so that the number of dununy 
pixels needed is 202 mod 4 (the rcmiundcr of 202/4). which 
equals 2. The microprocessor computes the number cf 
dummy pixeb required based upon the current position of 
the video window and stores the numba in the frame 
grabber controller. Eadi time the video window is moved cs 
resized, the microprocessor must update the number d 
dummy pixels in the frame grabber controller. The frame 
grabber controller 52 provides the dummy pixel value (i.e. 
the number of dummy pixels to insert) to the dimimy pixel 
logic 53 (described below), whic^ controls the clocking in of 
data from the decoder/sealer 44 to provide the proper 
number of dummy video pixels before each horizontal scan 
line of video pixels. Source selection logic 4S uses the pixel 
codes from window-type memory 38 to disable the proper 
tri-state buffers on the outputs to the video memory. 

In the described embodiment dununy pixels are created 
and inserted in tbe video memory by ''clocking** identical 
video pixel values into multiple banks of tbe video memory 
46 at the same time according to a clock signal (preferably, 
the video pixels are first clocked into shift registers and then 
stored into the banks of the video memory from the shift 
registers, as described above). The Banks that are clocked 
depending on the number of dummy pixels to be inserted is 
shown in Table 1. 



TABLE 1 



Numbe- of Dutnmy Wideo Pixek 




to be Inserted 


Banks cbdud 


0 


A 


I 


A,B 


2 


A,B.C 


3 


A, B, C, D 



For example, if two dummy pixels are required as in FIG. 
dh, a single video pixel is clocked into Banks A. B, and C 
of the video memory simultaneously from decodei/scalea* 
44. The next video pixel from the decoder/sealer is clocked 
into Bank D. Only the video pixels loaded at Banks C and 
D will be selected to be output on video chanitels; the other 
two video pixels in Banks A and B are duplicate pixels of the 
pixel in Bank C (dummy pixels) that are lost Dimimy pixels 
are output in only one video pixel block per horizontal scan 
line, at the left border of the video window. 

Dummy pixels for the graphics memory are not needed, 
since the gr^)hics adapter chip typically generates graphics 
pixels for the entire screen and continuously outputs those 
fHxels frx>m the output buffer whether the tri- state buffers on 
the graphics memory are enabled or disabled. At the right 
edge of the video window, the source selection logic 48 
simply selects the prcpa graphics or video channels accord- 
ing to the pixel map in window-type memory 38. 

FIG. 4 is a block diagram of preferred diunmy pixel logic 
53 of the described eitxbodiment for controlling the insertion 
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of dununy pixels into die video (Hxel stream. Dummy plxd 
logic 53 {Hefcrably iochidcs a counter 120 and lane logic 
122. In the described embodiment the dummy pixel value 
that represents bow many dummy pixels to insert is a two-bit 

5 code that is written into the frame grabber controller 52 by 
microprocessor 31. This dummy pixd value is received by 
counter 120 on bus 24 before each incoming video scan line. 
Counter 12# is preferably a 2-bit modulo-4 counter that 
counts after each pixel is docked into video mem<Hy 46 

10 (known from the video dock 121 generated by the frame 
grabber controller). Counter 120 also receives a frame pixd 
indicator (FFI) signal on line 128. which dears counter 120 
to zero when the first valid video pixel of a scan line is input 
to the video memory. The FFI signal can be. for cxan^le. 

IS generated by a fl^fiop internal to the frame grabber con- 
troller whidi sets FPI high before the first video pixel on a 
scan line and dears FFI (low) after the first video pixel. The 
output (Q of the counter 120 is a two bit number output on 
bus 126 to channd logic 122. 

20 Channel logic 122 receives the 2-bil count C on bus 126 
and also recdves signal FPI on line 128. The FPI signal is 
set at the first video pixel on a scan line, and indicates to 
logic 122 whether or not the current pixd being docked into 
video memory 46 is the first (left-most) video pixel on a scan 

25 line, such as video pixel 114 of FIG. 36. Channd logic 22 
outputs four dock signals ACLK. BGLK, CCLK and DO-K 
on lines 130a-J. rcq>cctivdy. which are included in address/ 
control bus 90 coupled to video memory 46. Each of the four 
clock signak IdKkh-d is coujHed to an input Bank of video 

30 memory 46 as described above such that ACLK clocks a 
video pixel into Bank: A, BCLK clocks a video pixel into 
Bank B. etc. A video pixd is clocked into a particular bank 
by applying a high pulse signal on a corresponding dock 
signal line tMh-d. Thus, if a pulse signal is sent on line 

35 130a. a video pxd is docked into Bank A. 

Channd logic 22 implements the following logical equa- 
tions: 

ACXK=tFPI*(C=0>+FPI 



CClJC=!FPt*(C=2)+fPI»(C>=2) 
DCUK=!FPI»<C=3>rfn*(C=3) 

which indicate that for exan^e, ACLK is provided as a 
high pulse signal on line 130ia when FFI is false and C=0. 
or if FFI is true; BALK is a pulse signal on line 1306 when 
FPI is false andC=l, orFHistrue andCis 1.2 or 3; CCLK 
is a pulse signal on line 130c when FPI is false and C=2. or 
FFI is true and C is 2 or 3; and DCLK is a pulse signal on 
line UOfi when FFI is false and C=3. or FFI is true and C 
is 3. Logic 22 can be implemented with gates and other 
electronic components as is well-known to those skilled in 
the art 

FIG. 5 is a schematic view of a second embodiment 30* of 
display system 30. Id display system 30*. DAC 50 has been 
repiaced with a different type of DAC 50*. DAC 50' includes 
nmltiplcxing circuitry on the duq> such that video channels 
78 of bus 74 and graphics channels 76 of bus 62 are 
connected to inputs of DAC 50* instead of bdng physically 
connected together. A suitable DAC for use as DAC 50 is 
the Brookirce BT8S5 DAC. This q)eciflc model cannot 
support 24-bit video using 4-way multq>lexing as described 
above. Instead, in the described embodiment. 16-bit video is 
preferably used, in which 5 bits are used for the red poition. 
6 bits for the green portion, and 5 bits for (he blue poition 
of the signal Thus, a 20-bit bus 74 indudes four video 
chaimds of 5 bits per channd. 



40 BCLK=:tFPI*(C=l)fFW(C>=l) 



45 



50 
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DAC 50' preferably selects graphics and video channels 
internally. Therefore, no separate source selection logic 48 
or window-type memory 3S is required in the present 
embodinaenL To select the correct channels. DAC 50* 
requires the location of the video window. The micropfo- s 
cessor writes the video window location infomation to 
DAC 50*. This information can include the coordinates of 
the opposite comers of the displayed video window. DAC 
SO' knows when to switch from gr^hics channels to video 
channels and vice versa by using internal conc^nents to lo 
keep track of the displayed pixels. A counter can be used to 
count the column at which an inpuned pixel is being 
displayed, and another counter is used to similaily count the 
row of the displayed pixei Four registers of the DAC can be 
used to store the location of the video window, including the 15 
column of the upper left comer of the video window, the row 
of the upper left comer, the column of the lower right corner 
of the video window, and the row of the lower right comer. 
The DAC can compare the current counter values of the 
column and row of a displayed pixel with the values of the 20 
borders of the video window stored in the registers to 
determine if graf^cs pixels or video pixels should be 
selected and displayed. DAC 50' then enables or disables the 
piopei video and graphics channels according to intenial 
components, as is well-known to those skilled in art Using 25 
the video memory of the previously-ilescribed embodiment 
DAC 50' can display a 512x512 pixel video window. 

DAC 50' also preferably includes a small first in fint out 
(FIFO) queue. Video data can be buffered or stored in this 
FIFO until it is displayed. The FIFO thus eliminates the need so 
for inserting dummy pixels on the video channels by dununy 
pixel logic 53, since the video dau can be stored indepen- 
dently of displayed graphics pixels. The other components 
of display system 30* operate similarly to the components of 
display system 30. 33 

By using a separate video memory for video data in the 
present invention instead of sharing memory for graphics 
and video data, the graphics memory bandwidth can be 
much larger. A much higher data transfer rate can thus be 
achieved when displaying graphics pixels using this con- 40 
figuration. The use of **>r graphics channels and video 
channels allows data to be displayed at a rate fast enough to 
make full use of Che DAC*s high speed and avoid slowing 
down graphics-intensive operations. Finally, the use of 
dununy video pixels permits the video window to be located 45 
on any pixel column of the screen (i.e.. single pixel 
resolution) and not be constrained to a discrete boundary 
after a block of n graphics pixels in an n-way multiplexed 
system. 

Although only one embodiment of the present invention 50 
has been described, it should be understood that the present 
invention may be embodied in many other specific forms 
without departing from the spirit or scope of the invention. 
Particularty. the source selection logic of the present inven- 
tion can be implemented in many dififcrcnt ways to perform 55 
the function of selecting channels from graphics and video 
memories to display a video window with greater data 
bandwidth. 

The display system of the present invention has been 
described as applied to various specific implementations. 60 
However, it should be appreciated tiiat the described display 
system can be applied to a wide variety of applications. In 
some systems, for example, it may be possible to output 
specific pixels from victeo memory and align the video 
window widiout inserting dummy video pixcb. In other 65 
in^lemcntations. it may be desirable to add a third source of 
displayed pixels such that three different buses are multi- 



plexed b^QTC being input to die DAC. Therefore, the present 
examples and embodiments arc to be considered as illustra- 
tive and not restrictive, and the invention is not to be limited 
to the d^ails given herein, but may be modified within the 
scope of the appended claims. 
What is claimed is: 

1. A method for simultaneously displaying gr^hics data 
and video data on a display screen aS a conqiuter syacm 
having graphics menusy and video memory arranged to 
store image information to be displayed an the di^lay 
screen, the display screen displaying a multiplicity of pixels, 
wherein the gr^>hics memory and the video memory are 
each arranged to sequentially transmit blocks of pixel data to 
the display screen on ou^ut cfaaimels. and wherein each 
block of pixel data includes data for a plurality of screen 
pixeb that is transmitted simultaneously, the method com- 
prising the steps of: 

storing graphics data received fixxn a gr^>hics source in 

gr^hics memory; 
staring video data received from a video source in video 

memory; 

selectively outputting graphics data for a block of pixels 
simultaneously from the gr^ducs memory cm a number 
of graphics channels when only graphics data is to be 
transmitted to the screen on output channels connected 
to said gr£9>hics channels; 

selectively outputting video data for a block of pixels 
simultaneously from the video memory on a number of 
video chaimeis ccHrcsponding to said number of graph- 
ics channels when only video data is to be transmitted 
to the screen on said output channels, wherein said 
video channels are coupled to said graphics channels to 
form said ou^ut channels and wherein either graphics 
data or video data is output to a display screen on each 
ou^t channel; and 

when both graphics data and video data arc to be trans- 
mitted to the screen simultaneously in a single block of 
data on said output channels, selectively causing the 
output channels that are intended to carry gr^ihics data 
to transmit only graphics data and selectively causing 
the output channels that are intended to carry video data 
to transmit only video data. 

2. A method as recited in claim 1 wherein said step of 
selectively causing die output channels that arc intended to 
carry graphics data to transmit only graphics data and 
selectively causing die output diannels that arc intended to 
carry video data to transmit only video data includes out- 
putting graphics data from said graj^cs memory only on 
said intended graphics channels and outputting video data 
from said video memory only on said intended video chan- 
nels. 

3. A method as recited b claim 2 wherein said outputting 
ffaftxcs data from said graphics memory is accomplished 
by enabling, buffers at an output of said graphics memory, 
and wherein said outputting video data is accomplished by 
enabling buffers at an output of said video memory. 

4. A method as recited in claim 2 wherein said graphics 
memory includes an output buffer coupled to said gr^rfiics 
channels and wherein said video memory indudes an ou^t 
buffer coupled to said video chaimeis. wherein said output 
buffer stores pixel data which is output from said memory 
when said memory is instructed to next ou^t data. 

5. A ZQ^od as recited in claim 4 wherein said video 
mcmoiy includes a shift regista for storing pixel data, 
wherein a portion of said pixel data is shifted from said shift 
register to said ou^ut buffer when said inxel data is to be 
output 
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6. A method as recited in daim 1 wfaetein said number of to transmit only video data such that grf^cs data hem 
graphics channels and said number of video channels is four. said graphics memory is output only on said Intended 

7. A method as recited in claim 1 wherein said step of graphics channels and video data from said video 
selectively causing die output channels that are intended to memory is output only on said intended video channels; 
cany graphics data to transmit only graphics data and 5 

selecUvcly causing die output channels that arc intended to inserting a number of dummy video pixel yakes before 

carry video data to transmit only video data includes reading said video data in said diift register, said number of 

a memory ananged to store data indicative of which pixels P^^ being based on the position <rf a 

on said screen are intended to display graphics data and boundary separating graphics pixels and video pixels 

whidi pixels on said screen are intended to display video lO '^^^^T^ "^t ^?P"5 ' , 

*^ *^ 13. A method as recited in clami 12 wherein said step of 

8. A method as recited in daim 7 wherein said video data '"^8 .said dmmny video pad values includes clocking 
transmitted to said scrcco is 24-bit true color video data. "J^* P"*^ value into a plurality of pxel locauons m said 

9. A method as recited in daim 7. wherein said memory sl^ regujter of said video memory when said video data is 

. . ^ ,c received from said video source, such that said dummy 

is a wmdow-typc memory. t5 . . , , , _ ^ -j 

10. A meth^as recited in claim 1 further comprising a ""'^^ arenot ou^t from said video memory, 
step of receiving analog video signal from said video source ^. ^ ^ara^^ dispUying a video window on a 
and converting said W^o sigaal into digital video data and ^^^y * ^^^^^ compnsmg; 
storing said digital video data in said video memory. * graphics memory havmg a set of output graphics 

11. A method as recited in claim 18 further comprising a 20 channels suitable for simultaneously transmitUng 
step of scaling said digital video data to a predetcnnined gr^cs data for a plurality of saecn pixels; 

a video memory having a set of output video channels 

IX A method of simultaneously displaying grajAiics data suitable for simultaneously transmitting video daU for 

and video data on a display screen of a computer system f plurality of screen pixels, wherein each video channel 

having graphics memory and video memory, ananged to 25 coupled to a corresponding grq)hics channel to fwm 

store image information to be displayed on the display » of channels and wherein an output channel is 

screen, the display screen dispUying a multiplicity of pixels, coupled to each of said pairs of graphics channels and 

wherein the grap^cs memory and the video memory are video channels. 

each arranged to sequentially transmit blocks of pixel data to a selection element for selectively causing data from said 

the display screen on ou^ut channels, and wherein each 30 graphics memory and said video memory to pass on 

block of pixel data indudes data for a plurality of screen each of said output channels sudi that the output 

pixels that is transmitted simultaneously, the method com- channds transmit a block of pixel data that simuUa- 

prising the steps of: neously includes both grqihics data and video data 

storing graphics data received from a graphics source in (^yidc^ on a discrete pixd basis; and 

graphics memoiy; a converter element fcr converting data on said output 

f^^^ - *^a^ channds inio a form suitable for driving the display 

stonng video data received from a video source in video ^ ^ ^ ^ ' 

meraorv* screen of the con^utcr system. 

^* . . ^ . , . , . . IS. An apparatus as redted in claim 14 wherein said 

selectively outputung graphics data for a block of pixels ^ comprises a VRAM chip staing graphics 

simultaneously from the graj^cs memoiy on a number ^ ^ graphics diip. 

of graphics diannds when only graphics data is to be ^ apparatus as recited in claim 14 wherein said video 

transinitted to the screen on output diannds connected comprises a VRAM diip storing video daU 

to said graphics channels; ^^^^^ 3 ^^^^ 

selectively ou^tting video data for a block of pixels 17. An apparatus as redted in Haim 14 wherein said 

simultaneously from the video memory on a number of 45 selection element includes window-typ>e memory having a 

video channels corresponding to said number of graph- memory map of a location of a video window displayed on 

ics channels when only video data is to be transmitted a display screen of a computer system, 

to the screen on said output channels, wherein said ig, An fqiparatus as redted in daim 17 wherein said 

video channds are coupled to said graphics channels to selection element indudes source sdection logic (^>erative 

form said output channels and wherein dthcr graphics 50 to read said menKxy map in said window-type memory and 

data or video data is output to a display screen on each select said data on said output chaimels according to said 

output channel, wherdn said gr^>hics memory includes window-type memory. 

an output buffer coupled to said graphics diannds and 19, An apparatus as redted in daim 18 wherein said 

wherein said video memory indudes an ou^xit buffer source selection logic is coupled to said grq>hics mcmcry 

coupled to said video channels, wherein said output 55 and said video memory and is operative to enable output 

buffer stores pixel data whidi is output from said buffers of said graphics memory and said video memory to 

memory when said memory is instructed to next output select said data on said output channels, 

data, and wherein said video memory indudes a shift 20. An apparatus as redted in Haim 17 wherein said 

register for storing pixel data, wherein a portion of said converter element indudes a digital to analog converter 

pixel data is shifted from said shift register to said (DAC) coupled to said video memcry and operative to 

output buffer when said pixel data is to be output; convert a digital video signal from said video memory to an 

whta both graphics data and video data are to be trans- analog signal capable of being displayed on said display 

mitted to the screen simultaneously in a single block of screeiL 

data on said output diannds, sdectively causing the 21. An apparatus as redted in daim 20 further conqrising 

output channels that are intended to cany graphics data 63 a frame grabber controller coupled to said video memory 

to transmit only graphics data and selective^ causing and said source sdection logic for controlling the output of 

the output chaimels diat are intended to carry video data video data from said video memory. 
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22. An apparatus as recited in claim 14 wherein said set 
of graphics channels and said set of video channels each 
iodude four channels. 

23. An apparatus for displaying a video window on a 
display screen of a computer systeau conqxising; s 

a graphics memory having a set of output graphics 
channels suitable for simultaneously transmitting 
graphics data for a plurality of screen pixels; 

a video memory having a set of output video channels 
suitable for simultaneously transmitting video data for 
a plurality of screen pixels, wherein each video channel 
is coupled to a corresponding gr^>hics channel to form 
a pair of channels and an output channel is coupled to 
each of said pairs of graphics channels and video 
diannels. and wherein said video memory stores a 
□umber of dummy pixels positioned before said video 
data in said video memory such that said dummy pixels 
are output on video channels that are not selected to 
output to said converter element; ^ 

a selection element for selectively causing, data from said 
graphics memory and said video memory to pass on 
each of said ouqput channels such that the output 
chaimds transmit a block of pixel data ttiat simulta- 
neously includes both gr^hics data and video data ^ 
divided on a discrete pixd basis; and 

a converter dement for converting data on said output 
diannds into a form suitable fcs driving the display 
screen of the computer system. 

24. A computer system con^hsing: 30 
a processor, 

a memory clement coupled to said processor; . 

a graphics adapter coupled to said processor for receiving 
commands from said processor and outputting graphics 
data according to the commands; 

a gr^)hics memory coupled to said graphics adapter for 
storing said graphics data and having a set of output 
graphics channels suitable for simultaneously transmit- 
ting said graphics data for a plurality of saeen pixels; ^ 

a video converter for converting a video signal from a 
video source to video data suitable for storage in video 
memory; 

a video memcvy coupled to a said video converter for 
staring said video data and having a set of ou^ut video 45 
channds suitable for simultaneously transmitting video 
data for a plurality of screen pixels, wherein each video 
channel is coupled to a corresponding graphics channd 
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to fonn a pair of channels and wherein an output 
channd is coupled to each of said pairs d graphics 
channds and video channels; 

a selection element for sdectivdy causing data from said 
graphics channds and said video channels to pass on 
each of said output diannds such that the output 
channels transmit a hlocl: of pixel data that sinuilta- 
neously includes both gr^hics data and video data 
divided on a discrete pixd basis; and 

a converter eknoent for converting data on said output 
channels into a form suitable for di^laying said data; 
and 

a display screen coupled to said converter element opera- 
tive to display said converted data. 

25. A computer system as recited in daim 24 wherein said 
video converter indudcs an analog-to-digital converter 
(ADC) for converting an analog video signal to a digital 
signal. 

26. A computer system as recited in daim 25 wherein said 
video converter includes a decoder/sealer coupled to said 
ADC operative to convert said digital signal output from 
said ADC to video data suitable for storage in said video 
memory. 

27. A computer system as recited in daim 24 wherein said 
gra{^ucs memory con^aiscs a VRAM chip storing graphics 
data from a graphics chip and said video memory conqmscs 
a VRAM ch^> storing video data recdved from a video 
source. 

28. A computer system as redted in daim 27 wherein said 
video VRAM chip includes a tri-port VRAM chip having 
two sequential access ports and one random access port 

29. A computer system as redted in daim 24 wherein said 
sdection dement includes window-type memory storing a 
memory m^ locations of said pixels of video data 
displayed on said co^^)Utcr screen. 

30. A cQ^^>utcr system as redted in daim 29 wherein said 
sdection clement indudes source selection logic operative 
to read said memory map in said window-type txtemory and 
sdect said data on said video channds according to said 
window-type memory. 

31. A computer system as redted in claim 30 further 
comprising a frame grabber controller coupled to said video 
memory and said source selection logic. 

32. A computer system as recited in daim 24 wherein said 
converter element indudes a digital-to-anaiog converter 
(DAC). 

***** 
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PATENT NO. :5,896,140 
DATED : April 20. 1999 

INVENTOR(S) :James O'Sullivan 

It is certified that error appears in the above-identified patent and that said Letters Patent 
is hereby corrected as shown below: 

Column 2, line 4, change "(KGB)" to -(RGB)- 
Column 3, line 41 , change "off to -of- 
Column 1 1 , line 13, change "iripuT to -input- 
Column 12, line 15. change "us.ed" to -used- 
Column 12, line 23, change "dumnmy" to -dummy- 
Column 14, line 61, change ''tie" to -the- 
Column 1 7, line 1 6, change "comer to --comer- 
Column 18, line 10, change "an* to --on- 

Column 18, line 54, change "enabling, buffers" to -enabling buffers- 
Column 21 , line 21 , change "causing, data" to -causing data- 
Column 21, line 32, change "clemenf to -element- 
Signed and Sealed this 
Twenty-third Day of November, 1999 
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